1. Technical Field
The invention is related to a system and process for correcting errors and losses occurring during a receiver-driven layered multicast of real-time media over a heterogeneous packet network such as the Internet.
2. Background Art
Real-time media, such as radio and television programs, are broadcast from a single sender to multiple, geographically distributed receivers, who have all xe2x80x9ctunedxe2x80x9d to that sender. Commonly, the signals are broadcast from the sender by a terrestrial antenna, but satellite and wired solutions also exist. For example, in cable TV, the signals are broadcast from a sender by propagating a voltage along a coaxial cable to receivers connected to the cable.
It is also possible to use the Internet infrastructure to broadcast audio and video information. This is typically accomplished using the Internet Protocol (IP) Multicast mechanism and its associated protocols. An Internet broadcast (or more properly, xe2x80x9cmulticastxe2x80x9d) is provided to the set of receivers who have first xe2x80x9csubscribedxe2x80x9d to the information. Specifically, through an announcement mechanism, such as a web page, a broadcaster announces the IP multicast group address to which it will send a particular broadcast. The multicast group address is just a special case of an ordinary IP address. However, unlike an ordinary address which is used to identify the xe2x80x9clocationxe2x80x9d of a receiver where data is to be sent, a multicast group address is used by routers in the network to identify data being transmitted on the network as part of the broadcast, so that it can be routed to a subscribing receiver (who will have a completely different address). The receiver""s address is not included in the broadcasted information.
A receiver subscribes to the broadcast by notifying the network that it wishes to xe2x80x9cjoinxe2x80x9d the multicast group. The subscriptions cause various routers in the network to update their states, to ensure that the multicast information eventually reaches the subscribers. At some point the sender begins to send packets to the specified address. When a router receives a packet with that address, it sends copies of the packet through each outgoing interface that leads to a subscriber. This causes the packets to reach the subscribers at some point, albeit with the inevitable packet loss due to network congestion and buffer overflow.
At a later point in time a receiver may unsubscribe for reasons that will be discussed later. This also causes the routers to update their states. If a router no longer has subscribers downstream from an interface, it stops copying the multicast packets to that interface. If a router no longer has subscribers downstream from any of its interfaces, then the router itself unsubscribes from the multicast group, and hence no longer receives (from upstream routers) multicast packets addressed to that group. This process is reversible, and dynamic. Receivers may subscribe and unsubscribe as many times as desired. Thus, information is propagated through the network only as necessary to reach currently subscribing receivers. The processes of subscribing and unsubscribing takes only fractions of a second, thereby network bandwidth is not wasted unnecessarily.
In the Internet, the channels between the sender and each receiver vary dramatically in capacity, often by two or three orders of magnitude. These differences in capacity exist because the data transmission rates associated with the connections to a particular receiver can vary (e.g., phone line capacity, LAN and/or modem speeds). This heterogeneity in capacity can cause problems in the context of an Internet broadcast of real-time audio and video information. For example, a particular receiver may not have the bandwidth available to receive the highest quality transmission that a broadcaster is capable of providing. One early attempt to cope with this problem involved broadcasting the audio and video data at different transmission rates to different multicast group addresses, with the quality being progressively better in the data broadcast at the higher rates. The receiver then subscribed to the transmission that suited its capability. However, this solution was very bandwidth intensive as the same information (and more) had to be repeated in each channel. To overcome this problem, an Internet broadcast can be transmitted via a xe2x80x9clayered multicastxe2x80x9d. In a layered multicast, audio and video information is encoded in layers of importance. Each of these layers is transmitted in a separate data stream. A data stream is a sequence of packets all transmitted to the same multicast group address. The base layer is an information stream that contains the minimal amount of information, for the least acceptable quality. Subsequent layers enhance the previous layers, but do not repeat the data contained in a lower layer. Thus in order to obtain the higher quality, a receiver must subscribe to the lower layers in addition to the higher layers that provide the desired quality. For example, a video signal can be layered into packetized data streams of 8 Kbps (thousand bits per second). Each stream is sent to a different multicast group address. A receiver can subscribe to as many streams as it wants, provided the total bandwidth of the streams is not greater than the bandwidth of the most constrained link in the network between the sender and the receiver. For example, if the receiver is connected to the Internet by a 28.8 Kbps modem, then it can feasibly subscribe to one, two, or three 8 Kbps video layers. If it subscribes to more than three layers, then congestion will certainly result and many packets will be dropped randomly, resulting in poor video quality.
It has been proposed that the congestion problem be addressed using a xe2x80x9cReceiver-driven Layered Multicastxe2x80x9d (RLM) scheme, where each receiver attempts to optimize its received quality by subscribing to as many layers as possible without incurring substantial congestion and loss. It does this by xe2x80x9ctest joins,xe2x80x9d in which the receiver tentatively joins, or subscribes to, the multicast group containing the next layer. If performance improves, the test join is made permanent. Otherwise, the layer is dropped, i.e., the receiver unsubscribes from the multicast group. In addition, if performance degrades at any point during the multicast (due to congestion in the network), the topmost layer is dropped. However, in complex network environments such as the Internet, there is often congestion along the path between the sender and receiver that is xe2x80x9cambientxe2x80x9d in the sense that it is due to cross traffic between other senders and receivers. Therefore, it is not always possible to eliminate all congestion along the path from a sender to a receiver by cutting back on the rate of transmission between the sender and receiver, i.e. , by dropping layers of the multicast. This presents a significant problem because in RLM the receiver attempts to drop multicast groups until there is little or no loss. When there is ambient congestion, this results in the receiver subscribing to few, if any, layers, which in turn results in sub-optimal video and audio quality.
Accordingly, there is a need for a system and process that can overcome the congestion issue and its concomitant packet losses, without eliminating so many multicast layers that the quality of the received audio and video information is unacceptable.
The present invention accomplishes this task by, in essence, augmenting RLM with one or more layers of error correction information. This allows each receiver to separately optimize the quality of the received audio and video information by subscribing to at least one error correction layer. Thus, a unique receiver-driven, layered, error correction multicast system and process is created.
Ideally, each source layer in a RLM would have one or more multicasted error correction data streams (i.e., layers) associated therewith. Each of the error correction layers would contain information that can be used to replace lost packets from the associated source layer. More than one error correction layer is proposed as some of the error correction packets contained in the data stream needed to replace the packets lost in the associated source stream may themselves be lost in transmission. These lost error correction packets can be picked up by subscribing to a second error correction layer, and so on until all (or an acceptable number) of the source packets have been replaced, or there are no more additional error correction layers available.
The decision as to how many broadcast source layers and associated error correction layers to subscribe to at one time is in essence based on the inherent packet loss rate of the network connection and the maximum bandwidth available to the receiver. The idea is to subscribe to as many of the source layers as possible or desired, while leaving enough bandwidth available to also subscribe to the number of error correction layers for each source layer that will compensate for the inherent packet loss rate of the connection and provide an acceptable audio and video quality. It should be noted that there is also an option not to subscribe to any error correction layers. For example, in networks where the inherent packet loss rate is very low, the optimum quality might be obtained by subscribing to additional source layers and no error correction layers.
The decision logic could be viewed as a one time decision made prior to receiving the broadcast. However, the packet loss rate, or even the available bandwidth could fluctuate during the broadcast. Accordingly, the decision logic could also be implemented dynamically in that the number of source layers and associated error correction layers subscribed to is reevaluated on a periodic basis to ensure the optimum quality is maintained throughout the broadcast. It should be noted that this may entail unsubscribing to the topmost layer (or layers) at least temporarily to maintain a desired audio and video quality.
Any audio and/or video layering process currently used in RLM could feasibly be employed in conjunction with the above-described system and process to create the source layers. Likewise, any conventional process currently used in RLM can be adopted for encoding/decoding and packetizing/unpacketizing the audio and/or video signal in the present invention. Further, in regard to the error correction layers, it would be possible to use any currently existing error correction technique appropriate for packetized audio and video signals. For that matter, any combination of such existing error correction techniques could be employed.
In addition to the just described benefits, other advantages of the present invention will become apparent from the detailed description which follows hereinafter when taken in conjunction with the drawing figures which accompany it.